What this changes

The current ShardingCodec decode path always materializes the entire outer chunk, regardless of how small the user's slice is. A request that touches one inner shard pays decompression for every other inner shard in the outer chunk — on a typical layout (e.g. 740 daily inner shards in one outer chunk) that's roughly a 700× decompression tax on partial reads, and zarr-python doesn't pay it.

This PR adds two layered fast paths for the sharding_indexed codec, both transparent to user code (arr[a:b, c] is unchanged):

1. In-memory partial decode (Codecs/V3/V3.jl). When the chunk's bytes are already in memory, only decode the inner chunks intersecting the user's slice, copying each intersection straight into the output. Skipped inner chunks pay nothing.
2. Storage-aware partial read (ZArray.jl + Storage/Storage.jl). When the storage backend supports byte-range reads, fetch only the shard index plus the bytes of the intersecting inner chunks. Skipped inner chunks aren't read off disk either.

Storage interface

Three new optional methods on AbstractStore, each with a safe default that falls back to the existing full-read path:

• supports_partial_reads(::AbstractStore) -> Bool (default false)
• read_range(s, key, byte_range) -> Union{Vector{UInt8}, Nothing} (default: s[key][byte_range])
• getsize(s, key) -> Int (default: length(s[key]))

DirectoryStore opts in (using seek + readbytes! and filesize). Other built-in backends inherit the defaults — they automatically use the in-memory partial-decode path with no behavior change.

Toggle

Zarr.enable_partial_shard_storage_reads[] (Ref{Bool}, default true), modeled on the existing concurrent_io_tasks::Ref. Set it to false to fall back to the in-memory partial-decode path even on stores that opt in — useful for A/B comparisons.

Pure-pipeline detection

The fast path is gated by a small helper, sharding_codec(p::V3Pipeline), that returns the inner ShardingCodec only when the pipeline is "pure" (no array→array codecs before, no bytes→bytes codecs after). Compound pipelines (e.g. transpose + sharding, or sharding wrapped by an outer compressor) keep the existing decode path unchanged.

Tests

Two new testsets, 39 new test cases:

• test/v3_codecs.jl :: "sharding partial-read fast path" — pure-vs-impure pipeline detection, in-memory partial path via DictStore, storage-aware partial path via DirectoryStore, the toggle round-trip, and fill_value over partial slices of unwritten outer chunks. Slice patterns include single-inner-chunk, cross-inner-chunk, full-chunk, cross-outer-chunk, and whole-array reads.
• test/storage.jl :: "Partial-read storage interface" — AbstractStore defaults via DictStore (including missing-key handling) and the DirectoryStore overrides.

Existing test suite stays at 2482 passes; this PR adds 39 → 2521 / 2521 passing.

Docs

docs/src/UserGuide/partial_shard_reads.md describes when the fast path applies, the storage-interface methods stores opt into, and the toggle.

Performance

Measured on a (67 syms × 127M seconds) float64 sharded archive (the layout zarr-python writes by default for 4-year second-resolution archives), one-day single-symbol query:

• Before this PR: 665 s (decompresses the full outer chunk per call)
• After this PR: 1.0 s (one inner chunk decompressed)

That's the in-memory + storage-aware paths combined for DirectoryStore. Other partial-read patterns see similar 50-100× speedups; full-shard reads are unchanged.

Compatibility

• Only the V3 sharding path is touched. V2 arrays, V3 arrays without sharding, and sharded V3 arrays whose pipeline isn't "pure" all run on the existing path with zero behavior change.
• Storage interface additions are purely opt-in via the safe defaults; no existing backend has to change.
• The toggle defaults to true for the perf win; setting it to false recovers the previous in-memory behavior exactly.

🤖 Generated with Claude Code